Text Search Example
Example illustrating using a text search for metadata.
Table of Contents
Step1: Create a variable for holding text search value.
Step 2: Set the number of results to return per provider.
Step 3: Set type of search to harvest top n results per provider
Step 4: Set the request domain URL for the geodex web service.
Step 5: Create a cell array to hold URL submission parameters.
Step 6: Make call to the geodex RESTful web service using webread.
Step 7: Explore results.
Step 8: Now set the type of search to harvest top n results across all providers.
Step 9: Set the new request domain URL for the geodex web service.
Step 10: Make call to the geodex RESTful web service
Step 11: Examine the results
Step 12: Now use the RESTful web service to query for metadata about a dataset.
Step 13: Now examine the returned metadata.
Step 14: Examine Metadata from opencoredata.org
Step 15: Examine Data from opencoredata.org
Step 16: Examine Metadata from opentopo.sdsc.edu
Step 17: Examine Data from opentopo.sdsc.edu
Step 18: Display Corners on webmap
Step 19: Obtain all Metadata from opentopo.sdsc.edu and create Word Cloud
Step 2: Set the number of results to return per provider.
Step 3: Set type of search to harvest top n results per provider
Step 4: Set the request domain URL for the geodex web service.
Step 5: Create a cell array to hold URL submission parameters.
Step 6: Make call to the geodex RESTful web service using webread.
Step 7: Explore results.
Step 8: Now set the type of search to harvest top n results across all providers.
Step 9: Set the new request domain URL for the geodex web service.
Step 10: Make call to the geodex RESTful web service
Step 11: Examine the results
Step 12: Now use the RESTful web service to query for metadata about a dataset.
Step 13: Now examine the returned metadata.
Step 14: Examine Metadata from opencoredata.org
Step 15: Examine Data from opencoredata.org
Step 16: Examine Metadata from opentopo.sdsc.edu
Step 17: Examine Data from opentopo.sdsc.edu
Step 18: Display Corners on webmap
Step 19: Obtain all Metadata from opentopo.sdsc.edu and create Word Cloud
Step1: Create a variable for holding text search value.
text_search_value = "carbon";
Step 2: Set the number of results to return per provider.
number_of_results = 100;
Step 3: Set type of search to harvest top n results per provider
search_type = "textindex/searchset";
Step 4: Set the request domain URL for the geodex web service.
% Set core domain name for request URL
domain_url = "http://geodex.org/api/v1/";
% Create request URL
request_url = domain_url + search_type;
Step 5: Create a cell array to hold URL submission parameters.
params = {'q', text_search_value, 's', '0','n', num2str(number_of_results)};
Step 6: Make call to the geodex RESTful web service using webread.
options = weboptions('ContentType','json','Timeout',15);
results = webread(request_url,params{:},options);
Step 7: Explore results.
first_result = results(1);
index = first_result.index;
high_score = first_result.highscore;
result_set = first_result.or;
top_result = result_set(1);
top_result_url = top_result.URL;
top_result_score = top_result.score;
output = string( ...
"Provider Index = " + index + newline + ...
"Number of Results = " + string(length(result_set)) + newline + ...
"High Score form Provider = " + string(high_score) + newline + ...
"URL of Top Result for Provider = " + top_result_url + newline + ...
"Score of Top Result = " + string(top_result_score));
fprintf('%s\n', output)
Step 8: Now set the type of search to harvest top n results across all providers.
search_type = "textindex/search";
Step 9: Set the new request domain URL for the geodex web service.
Please see http://geodex.org/swagger-ui/ for a complete description of the web service call formats.
% Create new request URL
request_url = domain_url + search_type;
Step 10: Make call to the geodex RESTful web service
Use webread to make the RESTful web service call.
results = webread(request_url,params{:},options);
Step 11: Examine the results
number_of_results = length(results);
urls = string({results.URL});
firstURL = urls(1);
first_result = results(1);
firstScore = first_result.score;
output = string( ...
"Number of Results = " + string(number_of_results) + newline + ...
"URL of First Result = " + firstURL + newline + ...
"Score of First Result = " + string(firstScore));
fprintf('%s\n', output)
Step 12: Now use the RESTful web service to query for metadata about a dataset.
search_type = "graph/details";
request_url = domain_url + search_type;
dataURL = urls(contains(urls,'iedadata'));
dataURL = dataURL(1);
params = {'r', dataURL};
results = webread(request_url,params{:},options);
Step 13: Now examine the returned metadata.
output = string( ...
"Dataset Webpage URL = " + results.URL + newline + ...
"Name = " + results.Name + newline + ...
"Alternate Name = " + results.Aname + newline + ...
"URI = " + results.S + newline + ...
"Keywords = " + results.Keywords + newline + ...
"Description = " + results.Description + newline + ...
"Citation = " + results.Citation + newline + ...
"Date Published = " + results.Datepublished + newline + ...
"License = " + results.License + newline + ...
"Dataset Download Link = " + results.Curl);
fprintf('%s\n', output)
Step 14: Examine Metadata from opencoredata.org
Use the first URL from opencoredata.org
metadataURL = urls(contains(urls,'opencoredata.org'));
metadataURL = metadataURL(1);
params = {'r', metadataURL};
results = webread(request_url,params{:},options)
Step 15: Examine Data from opencoredata.org
t = webread(results.Curl,weboptions('ContentReader',@(x)readtable(x)))
Step 16: Examine Metadata from opentopo.sdsc.edu
metadataURL = urls(contains(urls,'opentopo.sdsc.edu'));
metadataURL = metadataURL(1);
params = {'r', metadataURL};
results = webread(request_url,params{:},options)
Step 17: Examine Data from opentopo.sdsc.edu
The data were downloaded using the web interface at http://opentopo.sdsc.edu/lidarDataset?opentopoID=OTSDEM.092014.2871.1
It requires you to fill out your name and email address and select a region. The Lidar data are returned in a GeoTIFF file.
The Lidar data were provided by he University of Maryland and the Sonoma Vegetation Mapping and Lidar Program under NASA Grant NNX13AP69G.
We use functions from Mapping Toolbox to read the GeoTIFF file, display as a texture map, and display the boundaries on the webmap.
The file is named output_be.tif. The be means "Bare Earth" which is one of the selections in the web form.
filename = 'output_be.tif';
[A,R] = geotiffread(filename);
figure
mapshow(A,R,'DisplayType','texturemap')
demcmap(A)
Step 18: Display Corners on webmap
You can see indeed that the GeoTIFF image is in Santa Rosa.
info = geotiffinfo(filename)
x = info.CornerCoords.X;
y = info.CornerCoords.Y;
mstruct = geotiff2mstruct(info);
mstruct.geoid = referenceEllipsoid('wgs84','feet');
[lat,lon] = minvtran(mstruct,x,y)
lat(end+1) = lat(1);
lon(end+1) = lon(1);
webmap('usgstopo')
wmline(lat,lon)
Here is a snapshot of the webmap:
figure
imshow webmap_Santa_Rosa_USGS_Topo.png
Step 19: Obtain all Metadata from opentopo.sdsc.edu and create Word Cloud
This step reads all the metadata from oppentopo.sdsc.edu and creates a word cloud of the citations.
metadataURLs = urls(contains(urls,'opentopo.sdsc.edu'));
citations = "";
for k = 1:length(metadataURLs)
metadataURL = metadataURLs(k);
fprintf('Processing: %s\n', metadataURL)
params = {'r', metadataURL};
results = webread(request_url,params{:},options);
citations(end+1) = results.Citation;
end
figure
wordcloud(citations);